Generalists are selected when evaluated by sum
2018-04-24
Rating in Japanese] of multiple tests and multiple judges' scores will result in a person with better overall ability being rated higher than a person who excels in only some abilities.
On the right is the contour line when evaluated by the sum of the two test scores, and on the left is the contour line when evaluated by the maximum of the two test scores.
The evaluation of Mr. A, who scored 60 points in both abilities, and Mr. B, who excelled in one ability, are reversed.
https://gyazo.com/54c1330c1f88eea0bfbfcf5e3768567b
relevance
This is an evaluation strategy for cases where absolute evaluation scores, such as tests, are not given, but only rank ratings are obtained.
What prompted me to write this article
--When I hear the word "unexplored", I get the impression that the projects that are selected are all of a high level. Please tell us about the criteria for the selection of the submitted ideas. In the selection process for unexplored junior, each PM decides on the adoption of a project from the viewpoint of "whether or not I want to work with this girl". After screening the application documents, each PM conducts a 30-minute online interview with the applicant whose idea they are interested in, and decides whether or not the project will be adopted. Since each PM evaluates applicants from different perspectives and based on different criteria, there is not much overlap among PMs in the number of applicants they nominate for interviews. -- I was surprised. I had imagined that all the PMs would evaluate all the ideas and adopt the applicants in order from the top to the bottom, using a point scoring system.
Specialists should not be selected in such a way.
Ignore the bad and focus on the good.
On the right side of the above figure, when evaluating the sum of x and y, A (60, 60) has 120 points and beats B (100, 0)'s 100 points because B's -100 is not ignored when comparing A (-40, -40) and B (0, -100), which is a subtraction from 100 points.
In other words, using addition to combine multiple evaluations is a neutral method that is neither additive nor subtractive.
max(x, y) is an additive method that ignores bad points, min(x, y) is a subtractive method that ignores good points
PS
Summing is the same as take the average (the order is exactly the same because you just divide by a constant). Since the majority rule is to take the greatest sum of each person's 0/1 ratings, the "sum rating" at issue here relevance
---
This page is auto-translated from /nishio/和で評価するとジェネラリストが選ばれる using DeepL. If you looks something interesting but the auto-translated English is not good enough to understand it, feel free to let me know at @nishio_en. I'm very happy to spread my thought to non-Japanese readers.